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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 
Listing of Claims: 

1 . (Currently Amended) A hyperlinked broadcast system comprising: 

a video source providing video information for a video program including a plurality of 
consecutive video frames; 

an annotation system tracing tracking movement of two or more video objects appearing 
in each of the plurality of consecutive video frames from a first location to a second location and 
generating annotation data and annotation data timing information, the annotation data including 
only one mask for each video frame of the plurality of consecutive video frames, each mask 
being stamped with the corresponding annotation data timing information based on a frame time 
of the corresponding video frame, each mask including location and graphics data of two or more 
graphics images to be overlaid on the two or more video objects appearing in the corresponding 
video frame; and 

an augmented video information transmission generator receiving said annotation data, 
said video information, and said annotation data timing information, said augmented video 
information transmission generator generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information, and transmitting the augmented video transmission signal to a receiver, 

wherein said augmented video information transmission generator associates said video 
information with said annotation data using said annotation data timing information, 

wherein the receiver receiving the augmented video transmission signal is programmed, 
for each of the plurality of consecutive video frames, to: 

compare a current annotation data timing information with a current frame time of 
a current video frame; 
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retrieve the location and graphics data of the two or more graphics images from 
the identified mask if the comparison results in a match; and 

overlay the two or more graphics images on the corresponding two or more video 
objects appearing in the video frame based on the retrieved location and graphics data, 

wherein the overlaying of the two or more graphics images is synchronized on a frame- 
by- frame basis with the movement of the two or more video objects from the first location to the 
second location over the plurality of consecutive video frames. 

2. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises a vertical blanking interval insertion device. 

3. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises at least one of a vertical ancillary data insertion device and a 
digital video data multiplexer. 

4. (Original) The system of claim 1 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

5. (Original) The system of claim 1 wherein said video information comprises 
digital video data. 

6. (Original) The system of claim 1 wherein said video information comprises 
an analog video signal. 

7. (Original) The system of claim 1 further comprising: 
a post production environment; and 



-3- 



Appln No. 09/943,583 

Amdt date April 3, 2007 

Reply to Office action of January 3, 2007 

a headend comprising said augmented video information transmission generator, 
wherein said video information and said annotation data timing information are combined 
by said post production environment and transmitted to said headend. 

8. (Original) The system of claim 7 wherein said headed is a cable headend. 

9. (Original) The system of claim 7 wherein said headend is a satellite headend. 
10-13. (Canceled) 

14. (Previously Presented) The system of claim 1 wherein the receiver 

displays said annotation data in response to a viewer request. 

15-16. (Canceled) 

17. (Previously Presented) The system of claim 1 wherein the mask 
comprises location information of the two or more objects in the corresponding video frame. 

18. (Previously Presented) The system of claim 17 wherein said 
location information includes a graphics location reference that represents a fixed relation to a set 
of pixels associated with each said object. 

19. (Original) The system of claim 18 wherein said graphics location reference 
includes an upper left most pixel in said associated pixel set. 

20. (Original) The system of claim 18 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 
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21. (Previously Presented) The system of claim 1 wherein the mask comprises 
location and shape information of the two or more objects in the corresponding video frame. 

22-34. (Canceled) 

35. (Currently Amended) A method of generating a hyperlinked video signal 
comprising: 

receiving video information for a video program including a plurality of consecutive 
video frames; 

tracking tracking movement of two or more video objects appearing in each of the 
plurality of consecutive video frames from a first location to a second location and generating 
annotation data and annotation data timing information, the annotation data including only one 
mask for each video frame of the plurality of consecutive video frames, each mask being 
stamped with the corresponding annotation data timing information based on a frame time of the 
corresponding video frame, each mask including location and graphics data of two or more 
graphics images to be overlaid on the two or more video objects appearing in the corresponding 
video frame; 

communicating said annotation data timing information, said annotation data, and said 
video information to an augmented video information transmission generator; 

synchronizing said video information with said annotation data in response to said 
annotation data timing information by said augmented video information transmission generator 
and generating an augmented video transmission signal comprising said annotation data, said 
annotation data timing information, and said video information; 

transmitting the augmented video transmission signal to a receiver, wherein the receiver 
receiving the augmented video transmission signal is programmed, for each of the plurality of 
consecutive video frames, to: 

compare a current annotation data timing information with a current frame time of 
a current video frame; 
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retrieve the location and graphics data of the two or more graphics images from 
the identified mask if the comparison results in a match; and 

overlay the two or more graphics images on the corresponding two or more video 
objects appearing in the video frame based on the retrieved location and graphics data, 

wherein the overlaying of the two or more graphics images is synchronized on a frame- 
by-frame basis with the movement of the two or more video objects from the first location to the 
second location over the plurality of consecutive video frames. 

36. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises a vertical blanking interval insertion device. 

37. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises at least are of a vertical ancillary data insertion 
device and a digital video data multiplexer. 

38. (Original) The method of claim 35 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

39. (Original) The method of claim 35 wherein said video information comprises 
digital video data. 

40. (Original) The method of claim 35 wherein said video information comprises 
an analog video signal. 

41. (Original) The method of claim 35 further comprising inserting said 
annotation data timing information in a vertical blanking interval of an analog video signal. 
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42. (Original) The method claim 35 further comprising inserting said annotation 
data timing information in a vertical ancillary data region of a digital video signal. 

43. (Original) The method of claim 35 wherein said communicating step 
comprises transmitting said timing information and said video information to a broadcast 
network and subsequently to said augmented video information transmission generator. 

44-45. (Canceled) 

46. (Previously Presented) The method of claim 35 wherein the mask 
comprises location information of the two or more objects in the corresponding video frame. 

47. (Previously Presented) The method of claim 46 wherein said location 
information includes a graphics location reference that represents a fixed relation to a set of 
pixels associated with each said object. 

48. (Original) The method of claim 47 wherein said graphics location reference 
includes an upper left most pixel of said associated pixel set. 

49. (Original) The method of claim 48 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

50. (Previously Presented) The method of claim 35 wherein the mask 
comprises location and shape information of the two or more objects in the corresponding video 
frame. 
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51. (Previously Presented) The method of claim 50 wherein said shape 
information is represented by a graphical overlay of each said object. 

52. (Previously Presented) The method of claim 50 wherein said shape 
information is represented by an outline of each said object. 

53-54. (Canceled) 

55. (Previously Presented) The system of claim 1, wherein the annotation data 
further includes a plurality of object data packets, the mask further including an identifier to an 
object mapping table included in one or more of the plurality of object data packets, the object 
mapping table including at least one entry with an indicia from the corresponding mask 
identifying a particular video object, the entry further associating the indicia to information data 
structures included in one or more of the plurality of object data packets, the information data 
structures including information for the particular video object. 

56. (Previously Presented) The system of claim 55 wherein the to: 

retrieve the identifier of the object mapping table from the corresponding mask 
responsive to a user selection associated with the overlaid graphics image; 
retrieve the object mapping table based on the retrieved identifier; 

identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid; 

locate the entry in the object mapping table with the identified indicia; 
identify the information data structures associated with the located entry; 
retrieve the information in the identified information data structures; and 
display the retrieved information on the display device. 

57-60. (Canceled) 
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61. (Previously Presented) The system of claim 1, wherein the overlaying of 
the two or more graphics images is for alerting a viewer of interactive data associated with the 
two or more video objects prior to the viewer transmitting an interactive command with respect 
to one of the two or more video objects. 

62. (Previously Presented) The method of claim 35, wherein the overlaying of 
the two or more graphics images is for alerting a viewer of interactive data associated with the 
two or more video objects prior to the viewer transmitting an interactive command with respect 
to one of the two or more video objects. 

63. (New) A hyperlinked broadcast system comprising: 

a video source providing video information for a video program including a plurality of 
video frames; 

an annotation system generating annotation data and annotation data timing information, 
the annotation data including a plurality of masks and a plurality of object data packets, each 
mask corresponding to a particular video frame of a video program and including graphics data 
associated with one or more video objects in the particular video frame, each mask being 
associated with the corresponding annotation data timing information and with an identifier to an 
object mapping table included in at least a particular one of the plurality of object data packets, 
the object mapping table including an entry associated with each of the one or more video 
objects in the particular video frame, each entry in the object mapping table referencing one or 
more information data structures included in one or more of the plurality of object data packets, 
the information data structures including information associated with the corresponding video 
object; and 

an augmented video information transmission generator receiving said annotation data, 
said video information, and said annotation data timing information, said augmented video 
information transmission generator generating an augmented video transmission signal 
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comprising said annotation data, said annotation data timing information, and said video 
information, and transmitting the augmented video transmission signal to a receiver, wherein 
said augmented video information transmission generator associates said video information with 
said annotation data using said annotation data timing information. 

64. (New) The hyperlinked broadcast system of claim 63, wherein the receiver 
receiving the augmented video transmission signal is programmed to: 

compare the annotation data timing information for a particular mask with a frame time 

of a current video frame; 

retrieve the graphics data from the particular mask in response to a match; and 

overlay one or more graphics images generated based on the retrieved graphics data on 

the one or more video objects appearing in the current video frame. 

65. (New) The system of claim 64, wherein the receiver is further programmed to 
draw the one or more graphics images on a frame-by-frame basis based on the graphics data 
included in a plurality of masks, the drawing of the one or more graphics images being 
synchronized to the corresponding video frame based on the annotation data timing information 
associated with the plurality of masks. 

66. (New) The system of claim 65, wherein the receiver is further configured to: 
receive a user selection associated with one of the overlaid graphics images for a 

particular video frame; 

retrieve the identifier of the object mapping table from the mask corresponding to the 
particular video frame responsive to the user selection; 

retrieve the object mapping table based on the retrieved identifier; 

locate the entry in the object mapping table for the video object associated with the user 
selection; 

identify the information data structures referenced in the located entry; 
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retrieve the information in the information data structures; and 
display the retrieved information on a display device. 

67. (New) A method of generating a hyperlinked video signal comprising: 
receiving video information for a video program including a plurality of video frames; 
generating annotation data and annotation data timing information, the annotation data 

including a plurality of masks and a plurality of object data packets, each mask corresponding to 
a particular video frame of a video program and including graphics data associated with one or 
more video objects in the particular video frame, each mask being associated with the 
corresponding annotation data timing information and with an identifier to an object mapping 
table included in at least a particular one of the plurality of object data packets, the object 
mapping table including an entry associated with each of the one or more video objects in the 
particular video frame, each entry in the object mapping table referencing one or more 
information data structures included in one or more of the plurality of object data packets, the 
information data structures including information associated with the corresponding video 
object; 

communicating said annotation data timing information, said annotation data, and said 
video information to an augmented video information transmission generator; 

associating said video information with said annotation data in response to said 
annotation data timing information and generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information; and 

transmitting the augmented video transmission signal to a receiver. 

68. (New) The method of claim 67 further comprising: 

comparing the annotation data timing information for a particular mask with a frame time 
of a current video frame; 

retrieving the graphics data from the particular mask in response to a match; and 
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overlaying one or more graphics images generated based on the retrieved graphics data 
on the one or more video objects appearing in the current video frame. 

69. (New) The method of claim 68, wherein the overlaying further includes: 
drawing the one or more graphics images on a frame-by-frame basis based on the 

graphics data included in a plurality of masks, the drawing of the one or more graphics images 
being synchronized to the corresponding video frame based on the annotation data timing 
information associated with the plurality of masks. 

70. (New) The method of claim 69 further comprising: 

receiving a user selection associated with one of the overlaid graphics images for a 
particular video frame; 

retrieving the identifier of the object mapping table from the mask corresponding to the 
particular video frame responsive to the user selection; 

retrieving the object mapping table based on the retrieved identifier; 

locating the entry in the object mapping table for the video object associated with the user 
selection; 

identifying the information data structures referenced in the located entry; 
retrieving the information in the information data structures; and 
displaying the retrieved information on a display device. 
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